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ABSTPACT 

This^report outlines the procedures involved in the 
development of a test of literacy suitable for use in screening large 
numbers of persons. The authors discuss the problems which were faced 
from the initiation^ of the project t)aroiigh the final assembly of the 
test material^; describing ijie difficulty of definition, the 
practic&l constraints on^tlie administration, and the limitations of 
test design. OH +he fe^^s of i^s us$ thus far, the resulting 
instrui&ent, ref err'^d >.o" as ti(e "Brief Test pf Literacy," appears to 
discrimina*:e guic^kly and 4^u^2itely between 'literate and illiterate 
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/FC5REW0RD 

.The Health Examination Survey, one of the major programs of ihe 
. Nation^:enter for rfealth Statistics, collects, analyzes, andpubhshes 
the kinds of health- related data which can be obtained only through 
direct examinations, laboratory tests, and measuremerjj^Much of the 
data . collected pertains to prevalence levels of^,^fei5e|ific, medically 
defined diseases. Other data provide, fop^-^liej^^lation studied, 
distributions ,of a variety of physical, physiological, and psychological 
measurements. Reports in Series 1 arrd Series 11, described in the 
outline at the back of this, publication, present the descriptions and 
some of the findings of the various programs already carried out. 

In planning the third program of the series of Health Examination ^ 
Surveys, consideration was given to including some measure of the ex- 
tent of illiteracy in the population. That there is some relationship 
between various states of ill health and illiteracy has been recognized^ 
It seemed desirable, therefore, to be able to investigate the relation- 
ships between some of the health findings and this measure. In ad- 
dition, officials in other parts of the Department of Health, Education, 
and Welfare expressed interest in obtaining such data. 

The usual procedure followed in planning prograrfts of the Health 
Examination Survey is to utilize tests, procedures, and instruments 
already well established and generally accepted. In some instances, how- 
ever, the special requirements of the survey along with the "state of 
the art" of measurement 6f the 'particular variable make this kq- 
possible. This is discussed in the present publication. In this instance^^ 
presented with such a problem, it wds decided to enter into a contract 
with the Educational Testing Service to develop the required instru* 
ment. The results are presented in this report. 

k is not surprising that the National Center for Health Statistics 
should sponsor such research. The Public Health Service is authorized . 
under the National Health Survey Act (PL 652: 84th Congress) "to pro- 
vide (1) for a continuing survey and special studies to secure . . . eta^ 
listical information on the amount, distribution, and effects of illness 
Ind disability in the United States . . . and (2) for studying methods 
and survey techniques for securing such statistical information, with 
a vjew toward their continuing improvement." 

The results of this study are being made available, not only to 
provide' necessary information for evaluating later reports of findings 
In the Health Examination Surrey programs, but ajso because of their 
more general interest. The report will call attention to the need for 
technically superior, yet brief, psychometric instruments, and it will 
inform interested personis and groups as to what, has been done, in one 
instance, to meet this problem. 



Arthur Mcl3owell, Director 

Division of Health Examination Staristics 
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THIS REPORT outlines the procedures involved in the development of 
a test of literacy jxdtahle for use in screening large numbers of persons. 

fin it the abhors discuss the problems which were faced from the initia- 
tiov jf the project .through the final assembly of the test materials, de- 
scribing the difficulty of definition, the practical constraints on the ad- 
ministration, and the limitations of test design. 

On the basis of its use thus far, the resulting instrument, which will be 
} referred to as the Brief .Test of Literacy, would appear to discriminate 
quickly and accurately between literate and illiterate persons. This re- 
port -should jjrovide valuable information to any prospective user of the 
test or to those who seek to develop their oum instruments in this field. 
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INTRODUCTION " 

The Brief Test of Literacy was developed to 
assess literacy in reading and in writing within 
the framework of a national health survey. As §uch 
it provides an instrument of marked utility, for 
no prior test intended for the direct assessment 
of literacy has been developed. 

There are several reasons for the lack of any 
earlier development of a comparable instrument. 
In general, psychological testing has concentrated 
on the development of instruments which are 
appropriate for the measurement of individual 
difference&\with a concomitant interest in the 
longer tests that are necessary to achieve high 
reliability. Only recently has there been any strong 
interest in instruments that are specifically in- 
tended to provide information concerning the edu- 
cational attainment of groups. While instruments 
capable of such description will be developed with 
increasing frequency in the near future, the Brief 
Test of Literacy is' one of the first of its type, 

A second reason for the absence of a test of 
this kind is the concept of literacy. It is virtually 
impossible to achieve a satisfactory definition of 
literacy. It is evtn more difficult to attain an 
operational definition, and yiet an operational defi- 
nition is a virtual sine qua fwn for the develop- 
ment of a psychological test. The problem o( defi- 
nition is confounded by the varying demands of 
different cultures and subcultures and by cultural 
change through time. As a result, a person who is 



virtually illiterate by the standards of an advanced 
culture may well be able to meet the demands of 
his own less-developed civilization. ^ ^ 

A third reason for the absence of an earlier 
test of this nature'is that a large number of read- 
ing tests already exist. Many of these tests are 
intended to measure reading skill at approximately 
the level required. However, such existing tests 
can make a limited contribution to a survey of 
literacy because they are primarily , designed 
either to evaluate children who are in the first 
ye^rs of school or to provide diagnostic informa- 
tion concerning the nature of reading problems, 
rather than to provide categorical assessment of 
literacy versu^ illiteracy. 

For these'reasons, the Brief Test of Literacy 
represents an initial development both in the gen- 
eral field of survey instruments and in the assess- 
ment of literacy. 

GENERAL BACKGROUND 

The Brief Test of Literacy was developed for 
the purpose of assessing literacy in readfng and 
in writing within the framework of the National 
Health Survey whose mission is to study the inci- 
dence and prevalenc?fe of various health and health- 
related problems. Because of the nature of the 
survey, many different measures -are obtained 
for each sample person; therefojre, the amount of 
time aHotted for the assessment ofany one aspect 
of health ik extremely linrited* 

■ i 
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As a result one of the primary constraints 
plac<*d on the test was that It be so designed that 
literac> could be determined- in a brief period of 
time. Toward tKis epd a target time of 5 to 8 min- 
utes was established.. 

In addition to the time restraint, the test had 
to be suitable for use with the general population 
of adolescent^ throughout the continental United 
States and, hopefully, with adults as well. Since ^ 
the survey population excluded institutionalized 
persons, the test did not need to be designed tu 
permit the rapid as&essment of Jiteracy in ca^es 
wbere the individual could not function in normal 
feuCiety because 64 extreme emotional disturbance 
or severe mental retardation. 

A third constraint on the test was that it had 
to be S(5 designed that the results could ,J>e inter- 
preted in terms of the prevalence of literacy and 
of illiteracy. Accordingly, the fundamental fPf^^s- 
urement concept was that of a cutting score. Any- 
one above a designated score would be considered 
literate; those below it would be considered illit- 
ate. Degrees of Ikeracy would not be assessed. 

ESTABLISHING TEST ' \ 
SPEC^FlCATIONS^ FOR READING 

The initial step in the development of specifL- 

^cations consisted of a* survey of the literature. 

*This survey was disappqinting. In spite of exten- 
sive work on the importance of literacy and on 
projects for its improvement in;yatious nations, 
there w^reHo reports on techniques for its direct 
assessment. In fact, as stated in the introduction, 

' there is a general vagueness as to what consti- . 
tute^ literacy, with sundry definitions put forth 
by various writers. The mosl suirp^ising^ finely 
was the ^absence of any general 4escription oHhe 
assessment of literacy during World War 11. There 
undoubtedly was extensive work in the area It that 
time: the military differentiated among lowilevel 
inductees, determining who should be given a ba^ic 
education course, but there was nowhere a sum- 
nfia'ry of the devices used. From a p|rivate commu- 
nication with a government psychologist, it was 
learned that at present the Armed Forces use a 
general aptiude test to make these distinctions. 
JhjiS practice could not be followed by the survey, 

"(however, because of the obvious confounding of 
idw mentality and of illiteracy. 
\ 



While no specific techniques were uncovered 
in the literature search, a variety of definitions 
was'found. In general, these fell into *two classes, 
the functional and the normative. Functional defi- 
nitions stressed an individuals adjustment to his 
culture. One was litei^te if he possessed a level 
of ability sufficient to permit him to function well 
in his so^ety. Normative definitions stressed 
some^typl^l educational attainment. Thus, one 
was literate if one read as .well as the average 
child at the middle of the fourth ^ade in the United 
States, or at the end of the fifth year in Pakistan, 
et cetera. 

The functional definition is inherently attrac- 
tive, for illiteracy is a functional deficit. At the 
present time, however ,*^Jhere simply isnorealis- 
tic basis on which to determine a functional level 
for a society as diverse as that of the United 
States; to attempjt to describe the criterfa for 
using such a definition would be a truly formida- 
ble task. The following quot'ation of a UNESCO 
definition^ is an example of the difficulty. 



A person is literate when he Jias acquiredVie 
fssential knowledge and skills which enable 
htm to engage in all those activities in which 
literacy is required for effective functioning 
in his group and community, and whas^ at- 
tainments in reading, writing, and arimjxetic 
make it possible for him to continueio use 
these skills towards his ouinand the Commu- 
nity's development and for active participa-, 
tion in the life of his country. 

One would be hard-pressed to translate th^se gen- 
eralities into measurement specifics. 

Therefore, in conjunction with the adminis- 
tration of the survey for which the test was to be 
developed, it was decided to estimate Che incidence 
of illiteracy using a definition which is commonly 
held in the fields of education and health in this 
country, namely, "literacy is that level of achieve- 
ment which is attained by the average child in the 
United States at the beginning of the fourth grade ."^ 

With the establishment of a working defini- 
tion, the development of the statistical specifica- 
tions was begun. As stated earlier, the test was 
to be designed so that tes,t scores could be assigned 
to one of two categories. The, requirement built 
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in another bpecification— the ut>eof a cutting-bcore 
techriibue.^Given the working definition, the cutting 
score Nvwld ideally be. sujch^as ^to minimize the 
error in differentiating the top 50 percent of the 
national population of •children entering fourth 
grade from the bottom 50 percent. The item sta- 
tistics should be specified so as to achieve, then, 
this optimal putting score. 

. The theory of thjs cutting score ib quite com- 
plex. Major theoretical work in the area has been 
undertaken by Lord,^ and there are fairly sophis- 
ticated techniques for developing such tests and 
locating the "cut." For various practical reasons, 
however, a more pragmatic approach was used 
in developing the Brief Test of Literacy. "This 
, pragmatic approach did retain one obvious feature 
of virtually all cutting-score work^ the difficulty 
of the Items was centered on a narrow tiond/r^athtr 
than allowed to vary widely. This is in contrast 
to tests designed for differentiating among sev- 
eral levels of ability. 

A practical limitatidn also arose in connection 
with the timing of the deMelopmental work relati\e 
to the school yeaf/Thewprking definition of liter- 
acy was defined as, achievement at the beginning 
of the fourth grade, but the developmental work 
had to be performed during the late winter months. . 
If the scores made during Win^ei; months were to 
serve as estimates of the comparable difficulties 
which would be obtained using an entering fourth 
grade population, some adjustment in the obse^ eid 
item difficulties was nee^led.yrhere was, however, 
no adequate empirical* basis fot^ determining this 
adjustment. After a review of available data (jn the 
growth of reading ability, it was decided that-an 
average item difficulty level of/pO-percent-pass 
at the time of pretesting woulq5?e a useful esti- 
mate of a difficulty of 50 to 60 percent for enter- 
ing fourth graders. Accordingly, the specifications 
for item difficulty were set as follows: the items/ 
would show an average liifficulty of SO-percent/ 
pass and a range of diffl^culty from^65-percei)C- 
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pass to 95-percent-pass. 
^ , The difficulty of the 
specified to be approximate 
Deviations were pejcjrriitted ( 
"greater difficulty, because 
the niaterials with an older population fand be- 
' dause the normal conception o^ pleading difficulty 



adlng materials^ was 
fourth-grade /level 
iri the direction of ^ 
intended use of 
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lo based parrA on dimensions of reading be>ond 
the kind of literal comprehension whith was en- 
\isioned for, this test. This limitation to literal 
comjDrehension is discussed later in the descrip- 
tion of the type of questions asked. The conclu- 
sion was, however, that normal estimates of pas- 
sage difficulty were likely . to be overestimates, 
given the simplicity of the questions. 

In the jibs^ce of any external criterion, item 
validity was limited to an index of internal con- | 
sistenc>; phi coefficients^ were specified as the 
indexes of item-test correlation. No specific mesin 
value of these was established. Instead it was 
specified that the mean of.the phi coefficients be 
maximtzed and that all items should show a phi 
coefficient significantly greater than chance. 

The number of items in the test was also left 
unspecified. In a sense, there were incompatible 
goals for the proposed test in that test reliability 
had to achieve an acceptable level, while the time 
required fo\^ administration had to be minimized. 
A reliability between .70 and .80 was considered 
desirable for this survey work, andtheidea^test- 
mg time was 5 minutes per person. At the^ct^egin- 
ning, the format ofthe test was uncertain./Tlearly, 
there would be a presentation of mate/ial to ie 
read, and there woufd be questions to determine 
comprehension, but the severe time constraints' 
posed some difficulty in th^ development of test 
format. In any reading test there is usually an 
average ratio of the number of words which must 
be read for each question, .^is ratio must be 
large enough that aVeasonable test of reading can 
b&.attained, and it must be small enough that test- 
ing tirne can be efficiently used. The problem posed 
in the test development work w^s th^estimation 
of a workable value for this ratio. ^ 

CaorefulHtudy led to the conclusion that the 
optimum fornjat would consist ofa brief passage 
of 40 to 50 words followed by two gi; three ques- 
tions. Thus,' another speclfication^as established: 



of 



^Tho phi coofficiont a moa-^uro of corroladon 
U\oon ttto^sariahlo^ v\hon lhosanablc> aro^HsKiod intoquan ^ 
titativolj di^^croto groups and thu^ can brt ropro^ontod in a ^ 
four-fold table. It is identical to the productvnomont corrGla- 
tion botwoon twontbinomial vanato? The ph\ coofficiont i^ 
discus'^od in a numbor of .^tatistual to\ll)ODk\r furoxamplo, 
s<*o Walker and Vo\ . Stafatu al Infercru c. NpSl \grk. lU'nr> 
HoltandCo., Inc.. 1953 \ 
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the length of the pdb{>age to be read. Tht; Jecioiun 
wd& ait>o made tu ut>e three questions with each 
reading passage on the pretest. Ultimately, a de- 
cision would need to be made as to the use of two 
or three questions in the final form. This deci- 
sion could, be based both on the speed factor and 
on the patterns of losses of items due to defects 
uncovered in the pretesting. 

Timing was a central concern. Reading prg- 

»ficienc> has always consisted of a combination of 
two abilities: the ability to read rapidly and the 
ability to read accurately. Some reading tests 
attempt to provide diagnostic information as to 
Ihe relative proficiency along these two dimen- 
sions. Generally Ihe close correlation between the 
two measures, speed and accuracy (or compre- 
hension), poses no real difficulty. However, at the 
level of skill requiredto make a judgment o| liter- 
acy, less em^asis should be placed on speed as 

\|ie source of variation among scores. Certainl), 
in a functional sense, speed of reading is impor- 
tant in achieving literacy; ^nevertheless, many 

^poor readers must have time tc/allow the words 
CO come into focus before they can establish 

.nneaning. It*was obvious that, given the need for 
a ^minute test, no power measure could be pro- 
vide^. Every effort was made, however, to r^educe 
speed variance to a minimum. 

One underlying consideration in establishing 
time specifications was not essentially psycho- 
metric, but it was such a powerful consideration 
with those working on the test that it deserves 
nuention. "Illiteracy " is nota complimentary attri- 
bute, and although it is capable of specific redefi- 
nition in an operational sense— "an ^illiterate* is 
one who does poorly^orT^our test '—the popular 
conception of illiteracy cannot b,e ignored. This 
popular conception undoubtedly stresses compre- 
hension in reading far more than speed. In other 
words, to the extent to which it was possible, the 
test construction process limited speed variance 
to a level which seemed reasonable. The reading 
rates demanded by the test are not stringent in 
comparison with the everyday demands of our 
society. 

Since a random sample of noninstitutionalized 
persons aged 12 through 17 living in the continen- 
tal United States would be drawn in the survey, the 
typical sample subject should encounter no diffi- 
culty with the test. The poorer readers however, 
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fur whom there would exist a questignpf literacy , 
might have problems simply because of unfafnili- 
arity with any testing situation. The multiple- 
choice forrhat was specified for the reading test 
because of the efficiency it offered in response 
time .and in scoring time. The usCjOf a separate 
answer sheet, as opposed to a test booklet in which 
answers ajre recorded directly, posed certain 
problems. For example, a subject fnight fail to 
" correctly align his answer sheet and testbboklet, 
\t3Jiding to invalid test scores. However , since the 
use of an answer sheet made it easier for the ex- 
aminer totkeep track of the subject's progressand 
to stop the examination when the cut-off score was 
achieved, the answer sheet method was adopted. 

One concern remained. In a test of 5 minutes' 
duration, a butjjBtft of borderline intellectual abil- 
ity who is not used to taking tests might, if left to 
hir^elf, fail to divide his time properly. Thus he 
might spend too much timfe on, one particularly 
difficult question and thereby sCore poorly on the 
whole test. Such personal characteristics are a 
cause of concern even in much longer tests. Be- 
cause it was decided to avoid l^speededness" in 
all of its forms, personal characteristics seemed 
even more likely to cause difficulty. To c'bntrol 
for such variables, the test^as made to consist 
of a number of separately timed units, monitored 
by the examiner to insure that the appropriate 
pace was maintained. 

There were other reasons for developing a 
test of several parts. Foremost ampng these was 
the opportunity it would provide for shorteningthe 
* total testing time for any subject who succeeded 
in passing the cutting score. Such a subject could 
complete the part on which he ^yas working but 
would not need to attempt later parts. Another 
advantage would be derived ir; that an error in 
test administration during one of the parts need 
not require a complete retesting, rather, one addi- 
tional section could be added to replace the defec- 
tive one. The part^ were specified to be separately 
timed units, consisting of a passage and iwo or 
three questions. At this point no decision w^s made 
concerning'the amount of time which wpuld be de- 
voted to each passage; however, this was antici- 
pated to be 60or 90 seconds, depending on the out- 
come of^the pr,etesting. « 

The scoring formula w^as specified as the total 
number of right answers minus onerfourth of the 
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number ofi^rong answers. While this is standard 
practice in multiple-choice testing, it was partic- 
ularly indicated in this test, where the relatively 
few questions asked would make it possible to 
secure a substantial change in rank position 
merely by chance. If only the number of correct 
answers were used in the scoring, -^^^"V^ 
Specifications regarding the. content of the 
test were difficult to define. Perhaps the clearest 
specification was that the content had to be accept- 
able to adults and to adolescents, v^had to lend 
* itself to , pretesting on fourth graders. That is, 
materials from a storybook written for lO-year- 
olds would bfe inappropriate for adults, \)n the 
other hand, materials which would be pret^isted 
on a group of fourth graders could not contain 
language or topics inappropriate for children. In 
addition, niateriaU had to be suitable for use^ 
with highly diversified populations. For example, 
the test had to be equally acceptable to boys anS^ 
to girls, to person^ with a science interest and 
to those with an art fhterest, to those who lived 
in the country and to those who lived in the city,^ 
io^^gro and to white, and po rich- and pbor alike,' 
^Thej^^ticipated use of the te^t on older popu- 
lations led to the "pretesting of a number of pas^ 
^ sages aimed .at simulating' the functional reading 
demands of adult life. These were in the form of 
want ads and brief instructions for operating ' 
equipment, ^ ' 

One Important specification concerned the 
t>^pe of question whifch could be asked. In a reading 
test there is typically a variety of questions dif- 
ferentiated by the degrees of inference and judg- 
ement required to answer them correctly. Both ^ 
inference and judgment play a role in reading 
ability, and each may be argued to be essential 
to literacy, in one of its meanings. These more ' 
complex aspects of reading would be excluded 
from the definition of literacy used in developing 
this test. Instead questions would be limited to 
straightforward pdlr^p^^hension. As a result all 
answers would be essentially restatements of 
information presented in. t^e reading passage. 
While no defense of this decision may be neces-. 
sary. it may be restated? that any definition, of 
Uteracy is an arbitrary dichptomization of what 
fs fundamentally a continuum of varying reading 
3*ility from little or none, to highly developed, 
R^J&ding has dimensions, and it is possible to be 
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more literate inVne of these dimensions than in 
another. The most basic dimension in reading is 
straightforward comprehension, and th^ Brief ^ 
Test of ^.iteracy focused on this, j ^^^^..^^--^ 
When the foregoing work had been completed, 
^ the test specifications for the reading test were 
Virtually complete and the development of pretest 
materials was begun,. 

ESTABLISHING TEST 
SPECIFICATIONS FOR WRITING 

Very eariy in the development of the writing 
test the decision was made to use the technique 
of having the subject write a few brief, simple 
sentences in response to dictation by an examiner, 
Th^ writing test, because it called for a con- « 
kructeti response, required the development of 
^ scoring technique which would be efficient for 
the examiner, consistent when used by varying 
scorers, and valid in its differentiation anjong 
subjects, A central problem in developing this 
scoring technique was that of spelling accuracy. 
If a person writes "Kum kwik wid the dokter," it 
is difficult to say he is illiterate. On the other 
hand, not all variations in orthography are so 
readily translated, and it is difficult to judge 
when a message has been x^onveyed and when it 
has not. Similar remarks pertain to handwriting 
legibility. It was specified tha^the subject*s re- 
sponse 'could be either in printii^g or in cursive 
writing. Some highly literate persons produce a 
cursive script hi formidable difficulty. How could 
such products 'be fairly evaluated? 

^ It was decided that a two-dimensional ap- 
proach, incorporating both a summation of ^he 
^ correctness of particular words and a.-global 
Judgment of the sentence by the examiner woOld 
* be used. As stated below, however, this specifi- 
catipri \yas Subsequeijtiy abandoned on the -basis of 
jiretest results, ' ^ - , 

While the specification Of writing sentiences 
ars dictated was a prae^iciU ^ecisioh, its central 
importance should not be oWurlooked, Literacy in 
writing is typically conceived as tfie ability to 
produce, rather than reproduce, "a satisfactory 
message. Ideally, one would call for any. sort of 
written message from the subject, allowing .tlie 
subject to determin^its content. The message 
then would he evaluated* in some manner, ^Such • 

■ ' -- ' ■ ■ 



evaluations would be susceptible to vaijiation, how- 
iever. Furthermore thife would lead to a variety Of 
vocabulary samples since all subjects would not 
use the ^ame, words. Even worse, vocabulary 
content might well be selected bx the subject to 
insure his*success. 

/The^otal time required for the Voting test 
was left unspecified. The time allotted for writ- 
ing a given sentence was set at 1 minute, subject 
to modifipation following, the results of the pre- 
testing. Sentence length Was to be approximately 
10 words. Sentence topics were to stress practi- 
cal situations, such as instructions. 

The statistical specifications for the sen- 
tences could be \nore general, since the score 
\4iriance would be spread over more categories 
than the simple right- wrong.of the fnaltiple-cho*ice 
,*items used in reading. No precise difficulty meas- 
ure was specified; an index of con^stency with a 
total score and with the reading score was required 
but left unspecified until the nature of the scoring 
process was better definedt ' ^^ ^ * 

Consideration was given Vo -a format u^*^ich 
the subject would complete a brie^ document such 
as an application blank:. This^woj^ift^^na sense', 
the a'halog of the '^A^ant ad" passages v{hich were 
introduced intp the reading test. This format was 
rejected because it would produce responses which 
were uniqufe.to the ipdividual; what it offered in 
/face validity for evaluating adult literacy, it would 
Icjse in comparabili^^^ of subject performance. 

PREJESTS AND THEIR R^^ULf 

V In all, 25 reading passages and 75 questior 
were presented. The pretest population consis^teci 
^df 180 fourth-grade students selected from public 
.schools considered b^ the administrative officers 
of the school system to be about average in terms 
of national norms on ability tests. One minute was 
allowed for each passage and for its three ques- 
tions. Observation of. the first group cpnfirmed 
the appropriateness of this timing. The responses 
were iri^icated by circling the answer in the test 
booklet directly, rather than by use of an answer 
sheet, because the mastery of^an answer sheet is 
sometimes not complete among fourth-grade pu- 
pils and^ because the group administration proce- 
dure ysed in the pretest precluded the individual 
attjgjjtionHvhich could correct this. 



Table 1. Grade level and number of words 
used in each pretest ^ passage 




Grade level frequency, distribution 



"8.0-8.9— 
7;0-7.9— 
6. 0-6. 9-- 
5.0-''5.9^- 
4.0-4.9—- 



-1 
-2 
-3 
-6 
•13 



i Determined by Lqrge .formula, 

^"Adult" content (i.e., material from 
want ads or instruction manuals) 



The success of the pretest demanded that the 
Judgments of. difficulty be quite accurate. As a 
check on these judgments, the index^of reading ' 
difficulty proposed by Lorge^ was computed for 
each passage. This indei^ takes into account such 
factors as the length of the sentences and the num- 
ber of "uncommon** words (defined as any words 
not included in the Dale-Challlistof basic words). 

Data concerning this index and passage length" 
are presented in table 1. This table ^Shows that 



the average Lorge index was 5.4--that is, it 
corresponded in difficulty to the level of material 
with which the average pupil can cope in about 
the fourth month <^f the fifth grade. This figure 
was quite a bit higher than either the grade-level 
' index of the pretest population, which was 4.5, or 
that of the theoretical reference population, which 
was 4.0. It was felt that this was justified because 
the group for which the materials were being de- 
v^bped would be over U vears of age and there- 
fore, theoretically, beyo^ fourth-grade place- 
ment. Furthermore, th/e questions in the literacy 
test would be limited to assessment of compre- 
hension whereas the Lorge assessment was based 
on a complex of skill^^.. 

As stated in the discussions of the specifica- 
tions, there was an attegipt to develop materials 
with a higher "face-/validity" for adults, as illus- 
trated by items ftom want ads or instruction 
manuals which accompany appliances or equip- 
ment. In spite of efforts toward reducing the diffi- 
culty of this type of material, it constituted the 
seven most difficultpassages in terms of the Lorge 
index, 'as indicated by the high values associated 
with the passag^fe in table 1 which are marked 
with a dagger (/). Their possible value in securing 
subject acceptance was sufficiently great to war- 
rant^pretestin^t 

A In addition to the 25 reading passages , 10 sim - 
pie sentences' were read aloud, with instrucfeons to ' 
write ihem in the space provided. In general, there 
was more difficulty with the pretesting than had 
been «i\ticip^ed, for writing in response toxiict^- 
tion is not a routine school activity at this level. 
Fonunatel^. th^ true simplicity of the task made 
it possilp^e to elicit adequate responses with a 
minimum^ amount of assistance from^proctors. 

There wer^ three related statistics used in 
the evaluation ofthereading protest results. First, 
for eac^i question there was computed a phi coeffi- 
cient liieasuring its consistency 'with the total 
formula score on tfie entire ''5 qitestions for the 
whole group. Second, for each question* there was 
computed a phi coefficient, measuring it^ consist- 
ency wfth the total formula store for the bottom 
40 percent gf the total 'group^ Finally, for each 
passage the sum of thephicoeffitientsof its ques- 
tions, as determined on the bottom 40 percent, 
was computed. These statistical results are pre- 
sented in table 2. together with Information coij- 



cerning the level of difficulty of the material (in 
terms of the percentage passing;. As described 
in the footnote to this table, the phi coefficient for 
the total group is referred to as ; phi 20-80," and 
that for the bottom 40 percent as "phi 50-50^ ' 
reflecting the point at which the groups were di- 
vided. This P9int is, Of courbe, actually the same 
in both cases, for the 20th percentile in the total 
group is the 50th percentile in the lowest 40 per- 
cent. -Two different phi coefficients were required 
to insure effective differentiation of questions in 
the region of greatest interest. Appendix I pre- 
sents a more extended discussion of this. 

As indicated in tab^ 2, the pretesting was 
generally successful. Of''fhe25 passages, 12 se- 
cured a cumulative sum oK'phi 50-50 which ex- 
ceeded 100. Among the 36 questions which per- 
tained to these passages, only 4 had related phi 
coefficients which failed to attain statistical signif- 
icance at the .01 leveJ of confidence (phi equal to 
oiTgreater than .31), ^d 29 questions had coeffi- 
cients significant at the .001 level (phi equal to or 
greater than .39). 

Th^ passages with "adult" content were un- 
successful, with the exception of passage number 
22, largely because these passages were too diffi- 
cult to provide differentiation among the bottom 
40 percent. All of the 10 most difficult questions 
were associated with these materials. The gen- 
eral success of the difficulty estimation is indi- 
cated by the average cUffidulty of the questions 
which were not "adult" content. For these ^8 pas- 
sages, the average question was passed by 77 per- 
cefit of the total, ^oup. which was very near the 
specified value of .80.*One other point became 
clear in the pretesting. The third question was 
typically not much affected by "drop-out," the 
usual indication of *'speededness." Accordingly,' 
the use of ^ three questions with each reading 
passage could be continued in*<he final form. 

The writing pretest generally sustained. the 
appropriateness of the 1-minute time IJmit. The 
assessment of the consistency between success 
on a given sentence and success on a total score 
for writing (or. for reading) was not easy, as 
scoring procedures for the sentences had not 
been developed. Rough approximations were 
secl^red by scoring the sentences word by word.> 
the test of a word being the judgment that it was>. 
legible and that its meaning was conveyed in 



Tabl€f 2. Difficulty and validity indues 



Patsago and item 



I 

3 



7r- 
8— 
9-^- 



10- 
11- 
12- 



13- 
14- 
15- 



16- 
17- 
18- 



19- 
'20- 
21- 




28r 
29- 
30- 



31- 
32- 
33- 



34- 
35- 
36- 



37- 
38- 
39- 



M 
11 

11 



Total group 



Percent 
passing 

83 
63 
59 



90 
78 
81 



93 
75 
82 



88 
82 
69 



85 
45 
34 



ao 

76 
63 



79 
78 
64 



84 
83 
65 



86 
74 
63 



63 
76 
31 



84 
83 
78 



78 
75 
59 



89 
83 
82 



Phi, 20-80 

66 
65 
49 



53 
47 
47 



29 
38 
62 



45 
49 
45 



45 
34 
24 



46 
63 
30 



49 
50 
61 



36 
43 
69 



52 
76 
53 



25 
66 
21 



54 
67 
67 



53 
68 
49 



53 
56 
77 



Next- to-' 
lowest fifth 



Lowest 
fifth 



Percent passing 



Two lowest 
fifths 



Phi 50-50*^ 



Cusulative 
sum of 
phi 



Q1 




AQ 






A7 


25 


11 


18 


09 






01 


1 L 
1** ■ 


49 


72 


16 


56 








09 


/o 


20 


AO 




27' 


83 


33 


51 


0 J 


JO 


•27 


7S 




32 


42 


28 


15 


00 




31 


9«i 


11 


18 


22 


11 


15 


79 


LL 


28 


AO 


92 


47 


53 


33 


20 


AA. 
0** 


J7 


25 


AA 


^A 

JO 


• « 28 


39 


6 


40 


"7 C 


QQ 


lO 




AA 
OH 




39 




49 






42 


/J 


Q 

0 




36 


11 


30 






5 


Q1 


^ 1 Q 

17 


62 


6 


11 


-9^ 


«86 


44 


44 


86 


33 


54 


72 


22 


50 


72 


33 


39 


78 


17 


61 


42 


11 


35 


92 


56 


41 


81 


42 


40 


86 


22 


64 



96 
114 



• • • 

88 
144 



47 
98 



59 
74 



49 
64 



••• • 

75 
95 



53 
93 



• • • 

27 
76 



110 
140 



• • • 

67 
58 



• • • 

98 
148 



100^^* 



81 
145 



Table 2. Difficulty and validity irtdexes 



Passage and iteo^ 



Total group 



Next-to- 
lowest fifth 



Lowest 
fifth 



Two lowest 
fiJEthi . 



Cumulative 
sum of 
phi 



40- « 

41- . 

42- « 



43- 
44- 
45- 



46- 
47- 
48- 



49- 
50- 
51- 



52- 
53- 
54- 



55 

56 — 
57 



58- 
59- 
60- 



61- 
62- 

63- 



64- 
65- 
66- 



67- 
68- 
69- 



70- 
71- 
72- 



73- 
74- 
75- 



14 



ill 



17 



IS 



12. 



£2ft 



21. 



i21 



i2i 



.:^ercerit 
passing 

87 
64 
72 



70 
-57 
21 



80 
86 
80 



87 
75 
82 



89 
75 
76 



86 
86 
78 



26 
51 
22 



74 
66 
66 



81 
44 
53 



,53 
52 
45 



88 

78 
64 



41 
59 
28 



Phi 20-80^ 

56 
52 

' ' 65 



59 
49 
12 



148-^ 
6(>> 
72 



64 
48 
42 



66 
43 

.66 



67 
68 
63 



17 
18 

20 



50 
52 
46 



50^ 

39 

42 



28 
33 
29 



41 
61 
^6 



3X 
13 



Percent passing 



89 
36 
58 



61 
31 
6 



89 
97 
83 



97 
,69 
64 



97 
67 
78 



92 
^7 
75 



17 
39 
11 



61» 
53 
44 



89 
33 
42 



33 

25 
17 



83 
69 
33 



19 
33 
8 



50 
14 
14 



17 
8 
11 



42 
44 
22 



44 
19 
25 



47 
39 
19 



50 
39 
25 



11 
33 
6 



31 
17 
2i 



42 
6 
11 



25 
19 
16 



61 
28 

.19 



17 
22 
17 



Phi 50-508 

42 
25 
46 



45 
29 
-9 



49 
58 
61 



58 
50 
39 



56 
28 
59 



46 
62 
50 



30 
38 
23 



49 
34 
35 



25 
41 
16 



3 
12 
-14 



67 
113 



74 
65 



107 
168 



108 
147 



84 
143 



108 
158 



15 
24 



68 
91 



83 
118 



16 
17 



66 
82 



15 
1 





the total group Into the top 80 percent and the bottom 20 

the bottom 40 percent Into upper and lower halves # 

want ads ot Instruction manuals) , . " . 



A phi coefficient based on splitting 
percent. 

phi coefficient. based on splitting 
''"Adult*' contcAt (i.e., material from 



ERIC 



Table 3. Mean 



score .and range of scores, 
by fifths 



Fifth 


Mean 
score ^ 


Range of 
scores 


Highest scoring fifth-- 
Next-to-highest fifth- 
Middle fifth 

Next- to- lowest fifth — 
Lowest scoring fifth — 


20.47 
19.66 
17.47 
14.61 
2.92 


18-2X 
17-21 
. 12-21 
8-18 
-4-11 



^Mean score computed as follows: Total 
numbej? of correct answers minus one-fourth 
the number of wrong answers* 



lie 



context, ine distributions of these scores were 
then compared for the two lowQ^t fifths, using a 
rough "consistency measure' which counted the 
number of times that those in the next-to-lo\ve&t 
fifth on total score were better on the particu- 
lar sentence than tho^e in the lowest fifth, and 
vice vers'a. The^eater this dJfstance measure/ 
the more the agreement between the score for 
each item and the total score. Because the sen- 
tences were of unequal length, i^wever, they could 
not be readily compared. The labor of develop- 
ing the complex statistical information which 
would provide acomparisonwasnot justified. Vir- 
tually every sentence demonstrated a marked 
consistency with the total score; final selection 
was, in general, ^'based on other factors. A brief 
description of the consistency measure is pro- 
vided in Appendix 11.^ 

I 

CONSTRUCTION OF THE 

Fir^lAL FORM 

. On conclusion of the pretesting, the final phase 
of test specification and construction was under- 
taken. Of the 12 most successful passages, one 
passage (number 14) was eliminated because its 
cumulative phi depended greatly on the last ques- 
tion, raising the danger of ^'speededness." Then 
the pretest data were examined in order to deteJr- 
- mine an optimal number of passages for the final 
form. This Inumber was approximately seven, or 
21 questions. Accordingly, 7 passages were se- 
lected from the 11 possibilities. In this selection 
both item statistics and content were considered. 
Thus, pretest passage number 22 was preferred 



over passages with better statistics because of 
its "adult" content. The seven passages selectee^ 
are the first seven shown in Appendix IV. 

The total score characteristics of the seven- 
passage, 21 -item test were examined. Table 3 
present;^ the mean score using the formula, total 
number ^i)f correct answers minus one-fourth the 
number of wrong answers (R - i*W), onithe 21 
items for the ability groups defined by pretest 
items and the Icore range observed in each group. 

As shown, the test provides the greatest dif- 
ferentiation between the two lowest fifths and vir- 
tually none between the two-top fifths. This is, of 
course, the desired characteristic. An additional 
investigation of the separrtion between the two 
lowest fifths is provided by table 4^hich shows 
the score distribution for both. ^ 

The data in table 4 were the basis for the 
final decision concerning t^e^location of the cut- 
ting score, which was set at 10.75 or greater. 
That is, persons scoring 10.50 would be classed 
"illiterate," and those scoring lO.'^S would be 



Table 4. Ifermula score frequency distri- 
butions (R-1/4W) for the two lowest to- 
tal-score fifths 



Score 



-4— - 

-3— 

.-1--- 
0— . 

1 — 

2 — 
3-- 
4— 

5 — 

6 — . 
7— 
8 — 
9-- 

10-- 
11 — 
12— 
13-- 
14— 

15- - 

16- - 

lit 



•V" 



Next-to- - 
lowest fifth 



i^d2 

' 14' 

. " 'J. 

i?. 3 

f 11 



10 



4 

T^ble 5, Mean frequencies for and dif- 
ferences between the two lowest fifths, • 
'by response category 



Response 
category 


'Next-to- 
lowest 
fifth 


Lowest 
fifth 


Column 1 
minus 
column 2 


Total- 
Omitted-" ' 

Not reached-- 


Mean 
21.00 


frequen 
21.00 


cy 

... 


15.61 
3.39 

2.00 


. '5.92 
11.56 
(0.19 

— ^ 


.+9. 69 
-8.17 
<-0.'19 
-1.33 


Meai[i R-i/4W-- 


14.61 


2.92 


. ^ . 



classed literate" within the meaning of the work- 
ing definition. 

Table 5 pr^ents a comparison of the two 
lowest fifths withSrespect to the average number 
of responses whi'cTMbn into four basic categories: 
right, wrong, omitJed^and not reached. Both 

omitted' and ' noyreacaed' are blanks, with no 
response indicate^ on iht answer sheet. An '.pmit'^ 
is a blank which iafoUowd (not necessarily imme- 
diately) by a response to a subsequent question. An 
item is "not reachetr if it is left blank at the end 
of a series of responses. "Not readied" responses 
aro.used to indicate "|peededness"' in a test; 
' omit responses are generally considei^d to 
indicate ample time for a Response but a failure 
to petceive the correct response. There is always 
ambiguity about ths t<vo categories, an' omit' ma> 
not have been 3?ead, due to pressure of time; a 

not reached" item may have been considered. 
Nevertheless, the distinction offers some assist- 
ance in the quantitative assessment of apee^. 

As shown in table 5, there is a negligible 
amount of ' speededness' in the test. The differ- 
ence in score means between the two grbups is 
11.69; of this, only L33 is attributable to the dif- 
ference in nSt reached' items, and,then onl> if 
the lowest fifth can be assumed to have perfect 
success oil these items. In general, .then, "speed- 
edness' is a very sjiiall factor in the test. Further, 
the small number of ''onpits" indicates that the 



0 ^ . 



items are not skipped as the test is worked 
through. Apparently the salient characteristics 
of the items are such as to encourage responding. 

The reliability of the 2l-item test was esti- 
mated to be ,91 by a technique Suggested by Raju 
and Guttman.^ This estimate indicates an excel- 
lent reliability for the survey work for which the 
test is intended. Additional features of the test 
which heightened its utility for the survey were 
the use of the cutting score for securing briefer 
records and the availability of substitute passages 
for "repairing" a record damaged by the faulty 
administration of one of the passages. 

The final development of the writing test was 
broadly similar to that of the reading test. A 
five-sentence test-, totaling 47 words, with 1 min- 
ute per sentence was developed ^see Appendix Vj. 
The fijfe sentences were selected for appropriate 
cotTSTstency with a total-score criterion, for 
variety of content and vocabulary, and for sen- 
tence length. Once a scoring technique was de- 
veloped, a cutting score was determined. This i> 
between 2" and 28 (fractional scores are notpo&- 
sible): a persoiMgpring 2" is classed "illiterate' , 
a person scormg ^ 8 is "literate." This cutting 
score is estimated to divide subjects at grade le\ el 
4.0'ihto two equal groups on the basis of the data 
oTt the sample^of subjects at grade level 4.5. 

The principal labor concerning the writing 
test was the devising of a reliable scoring pro- 
cedure. Initial attempts to develop a scheme which 
relied on judgment for accepting or rejecting 
homophone approximations to standard orthog- 
raphy ("dokter," "tumorow^) proved u^iworkable. 
Even a^group of staff members acci/stomed.to 
working together on verbal items could hot secure 
a sufficiently high degree of consistency, i^fter 
much experimentation, it was decided to maximize 
the reliability ^ the scores by creating a scoring 
system which assigned a score based principally 
on errors of misspelling, of word inversion, and 
of word redundancy. This technique is described 
in the examiner^s manual ^Appendix VI). It has 
satisfactory Correlation with the various subjec- 
tive and judgmental .approaches, what it loses in 
oqcTaslonal instances by overpenalizing spelling 
eiTors, it gains in other cases by permitting dif- 
lerent raters to stere complex sentences in a 
'similar manner. 
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Table 6:r Length of time per test unit, 
basecf on performance of 12 sQ^dents 
identjified as poor readers 



Passage and 
Sentence 



I^assage 



1- 
2- 
3- 
4- 
5- 
.6- 
7- 
8- 
9- 
10- 
11- 



Sentence 



1- 
2-. 
3- 
4- 
5- 



SCREEN 



Average time 
in seconds 



47.5 
53.3 
43.1 
50.4 
42.5 
49.3 
50.3 
47.4 
46.3 
46.0 
47.3 



27.3 
.34.8 
33.3 
33.9 
38.4 



NG TRYOUTS 



The construction of the final form was fol- 
fewed by screening trvouts in whicH the new inscru- 
-rnent was administered in a person-to-person 
; situation to 24 studehts aged 14 through F who 
had been identified by reading teachers as havflig 
reading difficulty, Tme purposes of this tryout 
wece to assurfe that workable administration pro- 
cedures were\aeveloped and that passage content 
was equally acceptable at liie'older age range, and 
to check on ^'speedednass/' 

These trials wens very successful. While 
no formal validity estimates were prov^!&MTsiV the 
teachers, there was Informal evid^ce in that- 
the three persons who would be Judged "illiterate'^ 
by the test were in fadt so |udged by the school.)^ 
Expectations regarding the time element were '* 
confirmed. Even in ,thip population, there, was a 
considerable shortening of the time required as 
soon as any appreciable literacy was found. No 
use was made of the cutting' score, since all pas- 
sages needed to be screened for content accepta- 
bility, but tKe general ' practicality of the proce- 
dure was demonstrated.^ 



An answer sheet enabling all responses to* 
be recorded, both for reading and for writing, 
had been devised (see Appendix III). 

Table 6 presents the average time required 
for each passage and for each sentence as ob- 
served by one examiner in screening cryouts. The 
given averages are based on only 12 case's, but 
the consistency of the results across passages 
and sentences lends credence to thfeir reliability. 

These average times demonstrate that while 
the total working time for all tasks can be as 
much as 12 minutes, this will not often be the 
case. The Brief Testof Literacy is indeed *'briefJ' 

SUMMARY 

This detailed accgunt of ^e developmental 
procedures has concentrated on description rather 
than on critical evaluation. Many of the steps in- 
volved were based on assumption or professional 
judgment, the adequacy of these being crucial to 
the success of the enterprise. Similarly, where- 
ever statistical data were the basis for decision, 
the size of the sample from which they were 
drawn was a practical maximum rather than a 
theoretical optimum. 

Nevertheless, the general consistency qf the 
results and their coherence suggests that the 
developmental procedi^es were highly success- 
following the establish- 
completion of validation 
of Literacy will provide 
urvey purposes. 



ful. It is expected thai 
ment of norms and th^ 
studies, the Brief Tes 
a useful instrument for 
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APPENDIX I 

DISCUSSION OF THE USE OF PHI COEFFICIENTS 



The need for two phi coefficients, as presented in 
tjible 1. may be most quickly demonstrated by the 
following contrived examples of contingency tables. In 
each case the entries in cells and margins are per- 
centages of the total group, ^ 

The following question would show a sizable phi 
coefficient of consistency between item and test: 

TEST 



ITEM 

4 



10 


70 


80 


10 


10 


20 


20 


80 


100 



Suppose, however, that the performance of the top 
80 percent, in which seven-eighths or 87 ,5 percent were 
successful, was examined more closely as a 2 x 5 table 
in which each fifth of -the total group is presented* 
separately: 

ITEM 



10 


10 


20 


20 


20 


80 


10 


10 








20 


20 


20 


20 


bo 


20 


100 



Note that the item actually differentiates most 
markedly between the bottom 40 percent and the top 60 
percent. In fact, phi 50-50 on the bottom 40 percent 
would.be zero, correctly indicating that this item should 
not be chosen in spite of the value of phi 20^^80. 

On the other hand, there are anomalies in items, 
and it is the function of item analysis to guard against 



them. An item might show the following table for thfe 
4x)ttom 40 percent, which would yield a sizable phi: 

TEST 



ITEM 



5 


15 


20 


15 


5 


20 


20 


►20 


40" 



Information on ttie top 60 percent, however, might 
lea^ to a completed 2x5 table of 

TEST " 



ITEM 



5 


15 


5 


10 


20 


55 


15 


5 


15 


10 




45 


20 


20 


20 


20 


20 


100 



indicating that item ambiguity or some other peculiarity 
was distorting the normal pattern of increasing item 
success with increasing ability. For the 2 x5 table, phi 
20-80 would be computed from 



TEST' 



ITEM 



5 


50 


55 


15 


30 


45 


*20 


80 


100 



which would be lower tl^anj)hi 50-50, signaling the dis- 
torted pattern. ^**4,'v 

The fore'going cases are' necessarily preselected 
and dramatic. Nevertheless, the practice of usihg two 
coefficients of thi^ type in the development of a cutting 
score instrument has much to recommend it. 
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APPENDIX II 

DESCRIPTION OF THE COEFFICIENT OF SENTENCE CONSISTENCY 



An example of the consistency measure used In 
evaluating the sentences Is presented below. In general, 
a given sentence Is consistent with the total score If 
those In the more able group score higher than thosfe In 
the poorer group. The consistency measure totals the 
number of times that an Individual In the superlotlgfoup 
scores higher than an Individual In the less able group; 
from this total Is subtracted the number of times that 
individuals in the less able group surpass individuals 
among the superior group. 

Suppose that a given six-word sentence yielded the 
following distributions: 



4 

Sentence score 


« Tot^l score 


Lowest 
fifth . 


Next- to- 
lowest fifth 


6 * 










5 




5 
5 
10 
10 
10 


5 
10 
10 








5 




^ 5 








40 


40 



The consistency measure would-be computed as follows: 



, (1) 

Score 


(2) 

Number of 
superior group 
in category 


(3) 

Number of 
less able group 
they surpass 


\ 

Numbe^ of less 
able group who 
surpass them 


- (5) 

(2)x[(3)-(4)] 


6 


5 
•5 
10 
* 10 
5 
5 

• 


40 
40 
35 
30 
20 


5 

10 

• -r • 20 


200 
200 
350 
250 

* ' 50 
-50 

Index * 1 jOOO 








2 




♦ . ' ■* 



The chance expectation of this Index Is zero.nega- It Is similar \o several "such Indexes proposed In the 
*tlVe values'lndlcateian Inverse relationship,' et cetera. .L psychometric literature. ^ : ... ' 



. ^ APPENDIX III , ^ 
ANSWER SHErrS FOR TREADING AND WRITING TESTS 



8AMPZ2 PAGB 


HUM 


Question 

NuBiber 
» 


Ansvtr 

Choice 




Question* 

Nunbe'r 


Ansvtr 

Choice 




Question 


Answer. 

Choice 




















01 


A 


B C 


D ' B 






















« 




10 


A 


BCD 


E 


22 


A 


BCD 


E 


02 


A 


B' C 


D E 


























a 


A 


BCD 


E 


25 


A 


BCD 




0? 


A 


B C 


D E 


























i« 








2U 


A 


B C l/ 


E 










A 


BCD 


•e 








Question 
Nmber 


Antver 

Choice . 




















1 A 


B 


C D 


E 


13 


A 


B* C D 


E 


25 


A 


B /c D 


E 


2 A 


B 


C D 


E t 


lU 


'a 


BCD 


E 


26 


A 


A C D 


E 


5 A 


B 


C 


E 


15 

* 


A 


BCD 


E 


27 


a/ 

/ 


BCD 
• 


E 


U A 


B 


' C D 


E ^ 


16 


'A 


« C D 


S 


,28 


A 


B Q D 


E 


5 \ ^ 


B 


C I) 


E 


17 


A 


BCD 


fi 




A 


. 5 C D 

' \ 


E 


6 A 


B 


C D 


E 


18 


A 


' B C D 


I 


. 50-, 


A 


.BCD 


fi 


f % 

• 

7 .A 

8 - A 

9 A 

* 


\ 

B 
B 

' B 


. C D 
C D 
0 D 


E 
E 
E 


19 
20 
21 


A 
A 
A 


B C D E 

B e* D E* ^ 
B C D E 


51 

\ 

T 52 

* 

55 


A 
A 
A 


* 

BCD 
BCD 
BCD 


E 
E 

E 














• 


t 


* 









\ 



\ 
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APPENDIX IV 
INSTRUCTIONS FOk READING . 



ON EACH PAGE IN THIS BOOKLET THERE IS A 
SHORT PARAGRAPH WHICH IS FOLLOWED BY^THREE 
QUESTIONS. BELOW EACH QUESTION ARE FIVE 
STATEMENTS, ONLY ONE OF WHICH ^lAKES AGQOD 
AND SENSIBLE Al^TSWER. YOU SHOULD FIND THIS 
STATEMENT, AND MARK YOUR ANSWER BY CIR- 
CLING THE LETTER ON THE ANSWER SHEET WHICH 
CORRESPONDS TO THE STATEMENT YOU SELECT. 

YOU MUST WORK j\SQl)lCJCLY ASYOUCAMrF^Rr^fQU- 
WILL BE ALLOWED ONLY ONE MINUT^TO WORK 
OhJ EAQH PARAGRAPH. BECAUSE THE TIME IS SO 
SHORT, YOU MAY NOT FWISH ALL OF THE QUE S- 
TIONSu IF YOU DO FINISH A PAGE BEFORE THE 
TIME IS UP, TELL ME AND VOU WILL BE ALLOWED 
TO GO ON TO THE NEXT PAGE. , - ' • 



Department of Health, Education, and Welfare 
Public Health Service . 
National Center for Health Statistics 
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SAMPLE PAGE 

It was a beautiful gift, wrapped with bright red 
paper and tied with silver string. Itwas small, but ver> 
heavy. No one knew who had brought it, but it had iMr. 
Jones' name on top. Mr. Jones just smiled and said, 
^^Pll open it \*hen I get home." 

01. Whose name was on the top of the gift? 

(A) Mr. Jones 

(B) Mr. Pike . • , 
^ (C) Willy 

(D) The postman * ^ ^ 

(E) No one knew 

02. In what color paper was the gi/t wrapped? 

(A) Red * ' . • 

(B) Silver *^ 

(C) Green 
(]3) ^Orange 
(E)\ellow 

Where \yas the gift going to be opened? ^ 

(A) Where it 'was found 

(B) -T^t the police station • 

(C) In the car 



03. 



(D) At the office 

(E) At home 



DO NOT TURN THE PAGE 
UNTIL YOU ARE TOLD ^ 
TO DO SO. , . V 
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It was spring. The yqung boy breathed the warm 
air. threw off his shoes.' and began to run. His arms 
swung. His feet hit sharply and evenly against the ground. 
At last, he felt free. 

What time of year was it? 



1 



(A) 
(B) 
(C) 
(D) 
(E) 



Summer 

Fall 

Spring 

December 

July 



2, What was the young bdy doing? 



(A) 
(B) 
•(C) 
(D). 
(E) 



Running , ^ 
Jumping 
Going to sleep 
Driving a car 
Fighting 



3. How did he feel?^ 

(A) Hot 
^ (B) ' Free 

(C) Wy 

(D) Cold 

(E) Unhappy ^ 



erJc 



DO NOT TURN THE PAGE 
UNTIL YOU ARE TOLD 
TO DO SO. 
•1- 



There were footsteps and a knock at the door. 
Everyone inside stood up quickly. The only sound was. 
that of the pot boiling on the stove. There was another 
knock. No one moved: The footsteps ontheother side of 
the door could be he^^djnoving away. 

4. The peopl^ifiside the room 

(A) Hid behind the stove 

(B) Stood \Ip quickly , r 

(C) Ran to the door ' . 
(b) Laughed out loud 

(E) Began to cry 

5. What was the only sound in the room? 

(A) People talking 
(B>- Birds singing 

o (C) A pot boiling / ' • * 

(D) A dog barking 

(E) A man shouting 

6. The person who knocked at the door finally 

(A^ Walked in^o the room 

(B) Sat dowtjoutside the door 

(C) Shouted for help ^ 

(D) Wajked away . , 

(E) Broke dowD the door 

DO NOT TURN THE PAGE 
UNTIL YOU ARE TOLD • 
TO DO SO. 
-2- ■ 



Helen liked going to the movies. Sometimes sht 
went four times a week. E\^eryone said she was crazy. 
Why did she always- want to go out and spend mondy, 
they said, when she could slay home and watch tele- 
vision? 

7. * V^at did Helen like to do? * 

(A) She liked taeat 

, (B) She liked to swim 

(C) 3he liked tb watch baseball 

(D) She liked to watch movies 

(E) She liked to watch wrestling matches 

8. What did people think about her? 

(A) They thought she was crazy 

.They thought she was very smart 
(d) They thought she was very nice ♦ 
,(D) They thought she was ugly 
(E) They thought sh§ v^as very old 



9. What did people think she should do? 

(A) Write a book • . , , 

(B) W^tch television ^ \ 

(C) Go on a diet 

^ (D) Dye her hair 

(E) Stop talking so much 



DO NOT TURN THE PAGE 
UNTIL YOU ARE TOLD 
TO DO SO. " ' • 

- ' 19 



You could smell the fish ma-rket long before you 
could see it. As you came closer you could hear mer- 
chants calling out about fresh catches or housewives 
arguing about* prices. Soon you' could see the market 
itself; brightly lit and colorful. You coi^ld see fishing 
boats coming ip. their decks covered with silver-grey 
fish. 



10. 



What\kind of a market is described abQye? 

A vegetable market • 
A meat market S 



(.A) 
(B) 
(C) 
(D) 
(E) 



A fish market 
A flower market 
A fruit market 



11. What could you see coming in? 

(A) Tug boats 

(B) Rowboats 

(C) Passenger boats 

(D) Fishing boats 

(E) Sailboats . 

12. What covered the decks of the boats? 



(A) 


• t 

Rope 


(B) 


People 


(C) 


Cars 


(D) 


Boxes 


(E) 


Fish 



DO NOT TURN TO? PAGE 
UNTIL YOU ARE TOLD 
TO DO SO. 



-4- 



Dill settled down sleepily into the seat at the back 
of the bus. All he wanted to do was to sleep uAtil it was 
time to get off. But the noise of a nearby radio and the 
voices of the passengers kept him aW&ke. Without think- 
ing, Bi41 stood. up and Shouted, "Shut up, everybody!" 

. 13, In what was Bill riding? 

(A) A boat , ^ 

(B) A car * / 

(C) A plane 

(D) A taxi 

(E) . A;busi" 

14.,> What-dld BiU want to do as he rode? 



(A) 
(B) 
^(C) 
(D) 
<E) 



Sleep 

Eat 

Drink 

Talk* 

Read 



15. What did he shout? 



(A) 
(B) 
(C) 
(D) 
(E) 



"Help!" 

"This is my stopl" 

•^hut up, everybodyl" 

"Theresa a fire!" c> 

"We're going to crashi" 

DO NOT TURN THE PAGE 
UNTIL YOU ARE TOLD 
TO DP SO^ 
-5- 



Tiger is a large, yellow cat. At night he prowls 
uut&iJe and is ver> fierce. When he hears a noise, he 
lowerb his h^ad and walks with btiff legs. All the other 
cats are afraid tp come into his yard. 

16. When does Tiger prowl? 



(A) 
XB) 
(C) 
(D) 
(E) 



At dawn » 
At dinnertime 
In the afternoon 
In the morning 
At night. 



n. What doe©^ Tiger do when he hears anoise? 

(A) He runs away 

(B) He walks with stift legs • 

(C) He hides under xhe bushes' 
' , (0) He walks on tiptoe 

(E) He pretends he doesn't hear it 

18. Who is afraid to come into.his yard? 

(A) All the other cats 
J » (B) The dog next door 
'^1 ; (C) The people who live in the house 

(D) The mailman 

(E) Most of the birds 

DO NOT TURN THE PAGE 
UNTIL YOU ARE TOLD 
TO DO SO. 
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The model number of your radio is 'A -707. Weak 
sound may indicate weak batteries. Replace Mih fresh 
batteries. Failure of the radio to operate may indicate 
a loose connection. All connections should be checked. 
If the radio still does not work properly, take it to our 
service department, 17- B West I7th Street. 

19. Whfit is the model number of the radio? 

CA^>.707 

(B) 17-B 

(C) W-I7 

(D) B-17 

(E) AB-707 

20. What should be done if the sound is weak? 
(A) Use weak batterie? 

.(B) Sepd tiie model number to the service depart- 
ment 

(C) Replace the present batteries with fresh bat- 
teries 

(D) ' Check all the connections 

(E) Replace the connections 

21. What is the address of the service department? 



(A) 


17 


-A West 17th Street 


(B) 


17 


-B West 17th Street 


- (C) 


17 


-A West 7th Street 


(D) 


^A- 


707 West 57th Street 


(E) 


P 


-B West 5'^th Street 



DO NOT TURN THE PAGh 
UNTIL YOU ARE TOLfj 
TO DO SO. 
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Sara hated big^ dinners. There were so many dishes 
to wash afterwards, and no one ever thought to thank her 
for doing them. And people always stayed so late after 
a big dinner. Sometimes it was midnight before she 
could begin to clean up. 

22. Why did Sara hate big dinners? 

(A) Because she always ^te tpo much 

(B) Because people were so.noisy ^ 

(C) Because there were so manjw dishes to wash 
(Q) Because she was never invited' " • , 

(E) Because they were so expensive 

23. How often did people remember to thanlf Sara? 

(*A) Sometimes f * 

(B) Always 

(C) Never 

(D) Once 

(E) Twice ^ ' 

24'. How late did It sometimes get before Sara could • 
clean up? * , • 

(A) Noon ' ' / 

(B) ^^loFnlng - ' * 

(C) Afternoon * 

(D) Midnight 

(E) Evening ' ■ ' 



DO NOT TURN THE PAGE 
UNTIL YOU ARE TOLD 
TO DO SO. ■ 



l^know you are in there," said the sheriff. "You 
have five seconds to come out." 

"Come get mel" shouted the robber from 'inside 
the house. ' ' ' 

The sheriff began to count. "One. Two. Three." 
Suddenly, the robl^r walked out with his hands up. 

28. Where was the robber? 

(A) Inside the house ^ 

(B) By the river 

(C) In the bushes ♦ 

(D) On his horse., 

(E) In the barn 

29. How long did the sheriff give him to come out? 

(A) Five seconds* ^ • 

(B) One minute 

(C) Five minutes / 

(D) Ten minutes 
• (E)' An^hour 

30. What did the^robber do? 

(A) He ran out shooting both guns 

(B) He tried to escape and was shot down- 

(C) "Vie walked out with his hands up 

(D) He sneaked out and got away 

(E) He didn*t come out, so the sheriff had to 
go in and get him 

DO NOT TURN THE PAGE 
UNTIL VOU ARE TOLD ' 
TO' DO SO.- V- 
. -10- * ' 



.The cat brushed against the old man. He did not 
move. He only stood, staring up into the window of the 
house. The party inside l(¥>ked warm and friendly, but 
no one noticed him. The oid man walked sadly on. 
followed by the cat. ^ 

25. "^at kind of animal was with the old man? 

(A) Mouse ' 

(B) Dog 

(C) Horse 

(D) Cat ^ 
(£) Bird ^ ; 

' 26. What was inside the house? * 



(A) 
(B) 
(C) 
(D) 
(E) 



A party 
Some dogs 
An old lady 
A meeting 
A salesman 



27. The man is described as being 



(A) 
•(B) 
(C). 
(D) 
(E) 



Old 

Young 

Thin 

Fat 

Small 



ERIC 



. DO NOT TURN THE PAGE 
' UNTIL YOU ARE TOLD 
' TO DO SO. 
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His cigarette went out. His pen propped from his 
hand. His head began to nod. He was, all at once, asleep. 
Everyone in the room laughed, "for he had come to work 
only five minutes ago. 

31. What 4ropped*from his hand? ' 



(A) 
(B) 
(C) 
(D) 
(5) 



A pen , 

A pencil 

A piece of paper 
A telephone 
A book 



32. 



33. 



What was he doing after his head began to nod? 

(A) Talking 

(B) . Sleeping 

(C) Crying 

(D) Smoking 

(E) Leaving 

When had he come to work? ^ 



(A) 
(B) 
(C) 
(D) 
(E) 



3 



Half an hour ago 
Three hours ago 
Yesterday . 
Five minutes ago 
Forty minutes ago 

DO NOT TURN THE PAGE . 
UNTIL YOU ARE TOLD 
to DO SO, 
-lU . 
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.APPENDIX V 
• - FIVE ITEMS USED fN WRITING TEST 

1. Turn left at the next corner. 

2. School will be closed tomorrow because of heavy snow. 

3. Send today for your free copy of this book. 

4. If you need a doctor, call this number right away. 

5. Drop a dime in the slot and turn the jiandje to the left. 

O O 'C^- 



C. 
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BASIC SICILLS SURVEY 



READING AND WRITING 



- MANDAL FOR EXAMINERS 



© Copyri^t ,1966 ' 
by C 

Educational Testing Service 
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MANUAL FOR EXAMINERS 



INTRODUCTION 



ADMINISTERING THE READJNG TEST 



The Brief Test of Lit.eracy was intended co provide 
a sound basis for classifying subjects as "literate" or 
"illiterate" withm a very short time limit. There are 
two tests— one of reading and one of writing. The read- 
ing test contains seven brief paragraphs, each accom- 
panied by three questions, for a total of twenty-one 
questions; the writing test consists of five sentences 
totaling forty-seven words* 

The tests and testing procedures were designed 
to provide the maximum information for the simple 
categorical decision, "literate" or "illiterate," For 
both reading and writing, literacy was defined as ap- 
proximately that level of function which is attained by 
the average student at the beginning of the fourth grade, 
Smce the nature of the decision is essentially "either - 
or," a cutting score technique is used, all persons above 
^ a cenain test score are classed as literate, all persons 
below the score are classed illiterate. The cutting 
score in turn provides the basis for the very brief 
testing times which are possible with this instrutnent, 
for the testing need only be continued until this score 
IS achieved. That is, it is suff icier^t to be able to know 
that the subject is above the cutting score (hence "lit- 
erate by definition), how far above Is not important. 
Indeed, the instrument Is not well suited for differ- 
entiating among persons who are not near the cutting 
score. It tends to bunch such people into a single score 
category, since it has been specially built lo provide 
its maximum of information at and near the cutting 
score. In achieving this maximum, informatibn about 
differenp^ at other levels is necessarily losu 

Reqilired Materials 

An administration requires: 

(1) stopwatch 

(2) pencils (with erasers) 

(3) reading test booklet 

(4) answer sheets 

(5) manual for exaniiners 



Procedures 



Seat 'the* subject at a desk or table, provide him 
With a pencil, answer sheet and booklet, and have him 
write his name in the space provided. Then say: 



.This IS a brief test of reading and writing. It 
will last about ten minutes. Read the instructions 
on the cover silently to yourself while ^ ^d 
them aloud to you, 



Read as follows: 



On each page in this booklet there is a short 
paragraph which is followed by three questions. 
Below each question are five statements, only ofie 
of which makes a goo4 and sensible answer. You 
should find this statement, ^ndm^rk your answer 
by circling the letter on the answer sheet which 
corresponds to the statement you select. 

You must work as quickly as you can, for you 
will be allowed only one minute to work on each 
paragraph. Because the time is so short, you may 
not finish all of the questions. If you do finish a 
page before time is up, tell me and you will be 
allowed to go on to the next page. 



After reading the instructions, ask if there are any 
questions. The typical subject will NOT hfive any ques- 
tions; those who ^o will frequently merely require 
repetition of the appropriate part of the instructions. 
The following replies are suggested for possible* ques- 
tions in two areas: 

Erasing 

Questions: Can I change my answer? 
Is it o,k, to erase? 

Is it o,k, to cross out my first answer? 
Reply: Yes, but work as -quickly As you can. 



24 
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Guessing 

Questions: Is^c o.k. to guess? 

Do you'counc off for guessing? 
Can 1 g;uess? 
Reply: We are subtracting a penalty for each 
wrong answer, so wild guessing is unlikely 
to improve your score, and it may lower it. 
However, if you can eliminate one or more 
of the wrong answers, it is probably to your 
advantage to guess. 

AV'hen the subject Is ready, read the following, point- 
ing to the appropriate section of the answer sheet; 



Read t)ie)3aragraph and then answer the quest ions 
by circmig the appropriate letters (point to 01, 
02, 03 on the answer sheet;, There is only one 
correct answer for each question. Tell me when 
you have finished with the paragraph. 

Ready? Begin work. 



Begin timing. At the end of one minute, say: 



Stop working. The time is up. D^youhave ahy 
questions? 



if the subject completes the sample page in less 
than a minute, say: 



Finished? Fine.^ Do you have any questions? 



Few questions will be asked. Some may inquire 
about guessing or erasing, as described above; a few 
may wonder if the paragraphs in thet?stare any longer 
than the sample paragraph. A simple reply is. 

The paragraphs differ in length from page to 
page, but they are all about as long as this sample. 



"When -all is -r^dy^, say:. 



Now we will begin the test. Remember, if you 
finish a page before time is called, tell me that 
you are finished. °Do no: turn t<^ the next page 
until you are told to do so. 

Ready? Turn over tp page one and begin 
working. 



For each page, begin timing when the pages lie flat. 
* Some subjects will smooth the^booklet, others arrange 
, their answer sheet, they vary in the way they spend the 
first few seconds. Therefore, there is a need (pr a 
fixed starting point, and this is when the pages lie flat. 
Do not worry if individual subjects seem to take too 
long before beginning work, the time allotted is really 
quite generous and &ny capable readei has sufficient 
time to demonstrate his ability^. 



The remaining work of giving the test is repetitive. ( 
If the subject indicates that he is finished, say: 



Finished? Turn over to page and begin 

work. X 



Stop. Turnover to page — and begin work.^> 



Ahvays state the page number which the subject 
should be working on, in order to avoid confusiote 



Substitute Paragraphs 



The administration of a rapidly-paced examination 
often leads to errors in timing, etc, hi this examination, 
the time is so brief that a sneeze, a broken pencil, or 
other inadvertent interruption may cast dbubt on the 
performance on a given paragraph. For this reason, 
alternate passages are provided on pages 8-11 of the 
test booklet. If one of the initial seven passages must 
be replaced, it is suggested that it be done according 
to the following program: 



For Passage on Page 

. ■ i ' 
I ■ « 

5 
6 

7 ' ' 



Use Passage on Page 

8 
9 

11 
10 

9 

9 

9 



The same cutting score of 11 may be used in each 
'case. This procedure assumes an equivalence among 
passages that Is not rigorously true. However, it wDuld 
seem to be superior to the use of examiner judgment 
in effecting remedies for deviant records, for such 
Judgnients-are,5:haracterlstically unreliable. ^ 

Use of the Reading Test Cutting Score 

This test is scored by giving 1 point for a right 
answer, 0 for an omit, and -Ji for a wrong answer. 
The complete reading test consists of seven passages, 
with a total of twenty-one questions. Because of the 
penalty for wrong answers, the scores could range 
from '5k (all wrong) to 21 (all right). In practice, 
however, all that we are interested in knowing is 
whether or not the subject gets a "formula score" 
(R.3iW) greater than 10.5. If he does, he passes and is 
classed "literate", if his score is 10.5 or less, he fails 
and is 'illiterate" in terms of this test. The cutting 
score was selected on^the basis of the statistical in- 
formation concerning the test. 



If the subject does not finish in one minute, say: i 
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s 



Obviously, IfThfi-Subject completes the first four 
paragraphs and gees all questions correct, he has a 
score of. 12 and is "literate." There is ho need to give 
additional questions. Similarly, if he gets 11 right and 

I wrong, he will pass. Almost all capable readers will 
answer the 12 questions correctly, and in much less 
time than the four minutes allotted. Hence, the use of 
the cutting score can reduce the average testing time 
for reading, including instructions, to under five min- 
utes. 

To use the cutting score, the examiner must be in 
position to observe the subject's work unobtrusively. 
In effect, he scores the answer sheet as the subject 
works. This is^typically a simple operation and can be 
deferred until the fourth paragraph is begun. The scoring 
key IS provided on page 6 of this manual. 

Because the cutting score is between 10 and 11, it 
IS possible to accept the decision "literate" before all 
questions on the fourth paragraph are completed. It is 
also possible to accept the other decision, "illiterate," 
before the fourth page is completed. (In factj the de- 
cision "illiterate" may be reached at the conclusion- 
of the first three passages, if all of the nine answers 
to these passages are wrong, for even if the subject 
answered the remaining twelve questions correctly, he 
would fail to achieve a score greater than the cutting 
score.) It is recommended, however, that full «even- 
passage records be obtained for all subjects excepting 
only those wljp bav£ 11 or 12 right answers on the first 
four passages. ^ t 

This recommendation means- that even sublets 
\vho pass the cutting score in the course of their work 
on the fifth or sixt^i<passage, should be continued for 
the full seven passages. It awards a premium, in a 
sense,' to the perfect or near-perfect performance on 
the early paragraphs. Subjects ivho attain these ex- 
cellent records may be presumed to be so capable that 
near-perfect performance on th^ remaining questions 
may be granted. 

To summarize: the cutting score is between a^ 
formula score of 10.5 and one of 10.75; at 10.5 or less, 
the subject "fails" and is "illiterate," at 10.75 or 
greater, he "passes" and is "literate." Subjects will 
achieve the cutting score, or demonstrate an inability 
to achi^^e it at varying points \n^ their work. It is 
recommenSed, however, that all subjects complete all 
seven passages excepting only those subjects who ge? 

II or 12 right answers on the first four passages. The 
time saving of the cutting score will be realized for a 
very large percentage of the prospective group, ages 

^-17. Approximately 95% of this group may be antici- 
pated to, answer the twelve simple questions correctly 
and in a few shon minutes, f^r the remainder of the . 
group, the nbefl for a complete record is "more crucial 
and the attempt to save tln^e by shortening the record 
is not worthwhile. » , 
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^CORING INFORMATION 

■^'^ / Answer Keys 

Sample Questions 

01 A 

02 A 

03 E 

Test Questions (Pagts 1-7) 

Page 1 Question 1 C 

2 A 

3 B 

Page 2 Question 4 B 

5 'C 

6 D 

Page 3 Question J D 

S A 

9 'B 

Page 4 Question 10 C 

11 D? 

12 E 

Page 5 Question 13 E 

^ 14 A ^ 

15 C 

Page 6 Question 16 E 

17 B 

18 A 

Page 7 jQuestion 19 A * 

20 C ' 

21 B 

Supplementary Questions (Pages 8-11) 

Page 8 ' Question 22 C 

23 C 

24 ^ 

Page 9 Question 25 ^ D 
26 

27 A 

Page 10 Question* 28 A 
. " 29 A 
. / 30 C 

Page 11 Question 31* A . 

32 B 

33 D 

Decision Chare 

After four passages; 

Any subject having 11 or 12 right answers is 
classed "literate" and testing is discontinued. 

After seyen pagsages: ^ 

Any subject having 13 or more right answers 
. is classed "literate." 
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Any subject having 12 or more right answers 
and 5 or fewer wrong answers is classed "liter- 
ate.*' 

Any subject having 11 right answers and only 
1 or 0 wrong answers is classed "literate." 

All othef subjects are classed "illiterate." 



ADMINISTEIIING THE WRITING TEST 



As you say "Begtn writing," you should begin tim- 
ing. Allow one minute and then say: 



Procedures 



After the reading test is completed, say: 



That's the end of the reading test. The next 
test IS the writing test. Turn over your answer 
sheet. 



When the subject is ready, say 'the following, point- 
ing to the three lines of the first answer space at the 
appropriate time: 



Listen carefully, i am going to read a sen- 
tence to you and 1 want you to write it in the 
space provided after 1 have read it twice. Use 
as muc;h spaoe as you need, and tell me if you 
want the sentence repeated. You have one 
minute. 

Do you have any questions? • 



Most questions seem to be quasi -questions which 
repeat the instructions in different wording and merely 
require some simple confirmation. 

Example: "1 write down what you say?." 
Reply: "Yes." 

Some subjects may ask: "Do yoy count off for'poor 
spelling?"^ 

A suggested reply would be: "Yes, spelling does 
count, but just do the best you can." 

The questions which were mentioned earlier in 
connection with the reading test, concerning erasing, 
crossing out, etc., may also be asked at the beginning' 
of the writing test. Refer to the earlier-discussion for 
the suggested* replies. Still another question may con- 
cern the possibility of breaking the pencil. If this is 
asked, say: "If you break your pencil, 1 will give you* 
another/' " _ 

When all is ready, read the sentence twice at a 
moderate rate. As you finish, say: 



Begin writing. 



Stop. Listen carefully and 1 will read sen- 
tence (the next sentence). 



If the subject finishes before time is up, say: 



Finished? 1 will-re^ sentence^ 



In each case, say "Begin writing" as the signal to 
begin. 

The sentences are: 

1. Turn left at the next corner. 

2. School will be closed tomorrow because 
of heavy snow. ^ 

3. Send today for yotir free copy of this book. 

4. If you need a doctor, call this number right 
away, 

5. Drop a dime in the slot and turn the handle 
to the left. ; 



Some subjects will ask to have the sentence re- 
peated. Others may have an obvious difficulty but hesi- 
tate to ask. Th^ examiner should watch carefully and 
re^at the *sentence 9n his own muiative if the subject 
appears to need it. This is- not a memory test* No real 
harm can come from repetition. The average subject, 
of course, has* jio trouble retaining the sentence and 
would find further repetition an interruption. 

In general, the writing test can be completed re-' 
gardless of interruptions or Breaking of pencils, etc., 
for if the examiner wishes he can always instruct the 
subject to begin over again and time him from tlTe new 
start. That is, since the test is not' one of memory, 
practice makes little difference,. and a broken pencil 
or a fit of coughing or other interruption can be coped 
with by starting over again. If needed, the margins of 
the answer sheet will provide the space for a second 
attempi; on interrupted questions. 

SCORING INFORMATION 

The scoring of constructed responses always 
poses difficulties, largely because of the variety of 
deviations from -the norm which occur. Even in the sim- 
ple task used in this test, the poorest writers will pro- 
duce quite complex responses, difficult to evaluate. To 
reduce the problems and to secure reliability, the score 
for the wtriting test is based Sjmply on the number of 
words correctly spelled and on the correctness of their 
order* For example, the sentence 

f 

If yu need a dokter, call this nmbr rite away . 

rec-eives a score of 6^ 1 point for each correctly 
spelled (underlined) \CDrd>i^credirVis given for mis- 
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spellings, even when the approximations are as phoneti- 
cally acceptable as the words "yu," dokter," and , 
"rite," 

An immediate problem concerns the legibility of 

* the handwtiting. Inevitably, examiners will differ as to 
what the subject actually wrote. In general, the guiding 
pi-inciple should be to give the subject the benefit of the 
doubt on any given letter. That is, if the response to a 
word is so poorly written as to 'be meaningless, no 
credit for that word is given, but if S letter is unclear, 
do. not penalize. For example, if it is uncertain whether 
the subject really wrote "e" for "o" in "doctor," give 
the subject the benefit of the doubt and 1 credit for the 
word. To repeat: If a single letter is amlvguous, assume 
that it is correct; if whole words are illegible, do not 
give credit. 

The order of the words is important. The subject 
does not get credit even for a correctly spelled word 
if this word is out of place. For instance, if the example 
above had been written 

If yu need to call a^ dokter, use this nmbr rite 
away , 

the score for this would be 5, The subject would not 
receive credit for call, which is out of place. This 
second example also provides an instance of the intro- 
duction of new words into the response, for "use" and 
"to" do not appear in the original sentence. No credit, 
is lost for such introductions, which occur chiefly in 
the records of borderline subjects. The problem of 
determining the correctness of order is more difficult 
than is apparent at first glance. For example, one sub- 
ject responded to sentence 1: 

Turn next at the left corner, 

« 

To cope with the diversity of possible subject responses, 
the scorer writes above each co;rrectly spelled word the 
*number which indicates the order in which it appears 
in the sentence as dictated. For example: 

1 (5) 3 ® :? 6 
Turn next at tjie left comer , 

This receives a score of 4, according to the following 
procedure: No word is scored if its number is greater 
than the number of the word immediately on its right. 
In the example above, "next" and '*th6" would not be 
scored, for "5" is greater than "3*' and "4" is greater 
than "2," Because this procedure is basically mathe- 
matical and mechanical, it wiU not exclude the'same 
words as would a judge. In the foregoing example a 
judge would rule out "next" and "left," rather than. 
"next"» and "the," However, the same score is arrived 
at both by Judging and by applying the rule: four words 

• are given credit. In problems of more complex reorder- 
ings, the merit of- the mechanical approach will be 
apparent, for itv is quite simple and reliable, A device 
for tallying^ the eliminated words is simply to draw a 
circle around the numbpr aboye them. 



One difficulty arises from the tendency of border- 
line subjects to repeat words. Thus, one candidate 
wrote 

Send today for this free book t5day. 

This would be' scored as 5: - 

1 2 '3 (D 5 9 [H 
Send today for this free book today . 

There are two instances of the word today . When- - 
even this occurs, if you give credit for the first such 
wird, by the basic rule, draw a square around the num- 
ber of the second such word and omit it from further 
consideration in the scoriqg. By the normal application 
ofiljebasic rule, the word "book" should not be scored, 
for Its number, 9, is greater than the number of the next 
word. However, having credited the first "today," the 
second is deleted and does not affect the^value of ^'book." 

Some Examples of the Scoring . 

The following sentences were actually encountered 
in the testing: 

2 3 (2) 4 9 

Example 1 Sent today for fee cpy of your fee book. 

The score is 4, Do not credit "of" because the num- 
ber of the word to the right is less. Note that while 
actually "your" is more properly the misplaced word, 
the rule accounts for the inversion by excluding "of," 
The net effect is the same, . 

1 2'^ 4 7 8 [3 9 

. Example 2 Send today fou your charpy of tjiis of book. 

The score is 6, When the first "of is scored, note 
its position number, 7, and draw a^square around the 
second 7, This permits the word "this" to be scored 
when it is encountered later, for the^ next correctly 
spe^^led word is now "book" with a position number 
greater than that of "this," 

1 2 '3 4 5 6 8 
'Example 3 if you need a doctor call these number 
10 T 

^vrite away . 

The score is 8, Notice that punctuation errors, such 
as the failure to begin the sentence with a capital letter, 
ate not penalized, ^ 

Note : In sentence 5 of the test, the V<<ord "the" appears 
three times. Therefore, the oiJte for dealing with 
redundancy and misplacem^s cannot be applied^ 
to this word in this sentence. After assigning the 
number to each word, apply the basic rule. 

4 7 8 9" 11 12 

Example 4 Dron in and turn the hand to the • life. 



Hv' score is 6, Hie secondfappe^rance of "the" is 
•*iUndexed ^as 12, and isjiot considered to be a redundant 
expression of the earlier appearance in which the word 
vrag indexed 9/ 

.'^The following example was contrived lo clarify and 
demonstrate the scoring. 

^ 

^ Example 5 Send tuday for copy of yore free copy of 
* , . this book, 

Step 1 : Underline all correctly spelled worcjs. 

^ Send tuday for copy of yore free copy of 
this book . ~* 

Step 2: For each underlined word, write the num- 
ber which indicates its position in the original sentence « 

1 3 6 7 5 6 7 

Send tuday for copy of yore free copy of 
8 9 

this book . ^ 

Step 3 : Begin scoring, counting any word which ha^s 
a number less than that of the next correctly spelled 
word on its right. If you credit a word which appears 
twice, draw a square around its second appearance. For 
"example, the following sequence would be followed in 
the sample sentence above. 



Under- 
lined Number 
word 



Send 

foif 

copy 

of 



free 
cjopy 
of 
this 



Action 

* 

give credit 
give credit 

give credit; box redundant sec- 
ond "6" 

no credit, (next number, 5, is 
less than 7)-; do not box sec- 
ond "7" 

give credit 

no ctedit, because boxed 
give credit 
give credit 
give credit 



Step 4 : Total the number of credits to get score. 
Score would be 7. • ^ • 

Note that the rule is not harsh. The subject could 
score at most 9 credits. He is penalized only for the 



Tho ©xaminor must uso hi5 judgment in Msigning positional num- 
bers to tho word *»lhG*' in tho fifth sentence of tho lout, if there ate 
fewer Ihtn throe •*tho*s** tn tho rosponpo. He/e it sooms clear that tho 
first **tho" in tho origin*! sontcnco wts omitted b> tho subject tnd thtt 
tho "Hho's" in this response should be usslgncd tho positional num- 
bors of 9 tnd 12 rtther than 5 and 9. * 



miss()eUings, the only efiect of the rules about mis-.^ 
placement being to avoid an overcredit for^redundant 
correctly spelled words. Note also that a word isiiot* 
boxed in its second appearance if it is not credited in * 
the first place. 

Summary of Sc oring Rules 

~* ' \ 

*' . * ' 

(1) Score one point for each correctly spelied word 
if the positional number is not circled or boxed. 

(2) Circle any positional number which is greater 
than the positional number which next appears 
on the right. Ignore a posit^ional number ^^^hich 
is boxed. ^ < 

(3) Box any positional number which has appeared 
earlier with a worU which was credited. Do not " 
box a number if it was not credited in its 
earlier appearance, or if it as a second or third 
appearance of the word "the" in the fifth sen- ' 

- ten.ce of the test. 

(4) Do not penalize for punctu^ion. 

' (5) The" "score is ^e sum of all of the credited' 
words. ^ , V , 

The Cutting Score for Writing 

The cutting score for writing is set between 27 and 
28. Accordingly, a subject getting 27 is classed "illiter- 
ate"; a subject getting 28 is classed "literate,'' While 
it is possible to attain the cutting score before the five- 
sentences are completed, no decision based on shortened 
records, analogous to the four-passage decision for 
reading, is suggested, for the savingSrin time woulcj be' 
negligible and the complexity of the scoring process 
would place too great ^ burden on the examiner. 

Relationships between Reading and Writing; Reading the 
Best Single Index 

The^ Brief Test of Literacy produces two scores, 
each yielding a judgment "literate." Because these two 
scores are not perfectly correlated, some subjects may 
be judged "literate" by one test but not by the other. If 
it is necessary to determine the relationship between 
literacy^ and some otheor variable, the conflict in the 
status of these„dases must be resolved. On the basis 
of the available da^4 and logicajkonsiderations as to the 
nature. of the abilities, it is recommended that in any 
suqh cases the decision reached by means of the reading 
test be considered final. Thus, in relating literacy to age, 
for example, a subject who was "illiterate" in the light * 
of the reading test, but'literate" intermsof thcwritlng 
test would be classed "illiterate" in assessing the're- 
lationship in question. ' ** . 
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